CREATE: Multimodal Dataset for Unsupervised Learning, Generative Modeling and Prediction of Sensory Data from a Mobile Robot in Indoor Environments

The CREATE database is composed of 14 hours of multimodal recordings from a mobile robotic platform based on the iRobot Create. The various sensors cover vision, audition, motors and proprioception. The dataset has been designed in the context of a mobile robot that can learn multimodal representations of its environment, thanks to its ability to navigate the environment. This ability can also be used to learn the dependencies and relationships between the different modalities of the robot (e.g. vision, audition), as they reflect both the external environment and the internal state of the robot. The provided multimodal dataset is expected to have multiple usages: • Multimodal unsupervised object learning: learn the statistical regularities (structures) of the sensor inputs per modality and across modalities. • Multimodal prediction: learn to predict future states of the sensory inputs. • Egomotion detection: learn to predict motor states from the other sensory inputs (e.g. visual optical flow, gyroscope). • Causality detection: learn to predict when the robot affects its own sensory inputs (i.e. due to motors), and when the environment is perturbing the sensory inputs (e.g. the user moves the robot around, the robot sees a human moving). Copyright and Trademark Notice: iRobot, Roomba and Create are registered trademarks of iRobot Corporation. 1 ar X iv :1 80 1. 10 21 4v 1 [ cs .R O ] 3 0 Ja n 20 18